# Efficient Quantized Inference

This is a quantized version of Gryphe's Codex-24B-Small-3.2 model, which optimizes the running efficiency under different hardware conditions through quantization technology.

Qwen3 is the latest generation of large language models in the Tongyi Qianwen series, offering a complete combination of dense models and Mixture of Experts (MoE) models. Based on large-scale training, Qwen3 achieves breakthrough progress in reasoning capabilities, instruction following, agent functions, and multilingual support.

This project provides weighted/matrix quantized versions of the llava-1.5-13b-hf model, including various quantization types to meet the usage requirements in different scenarios.

A quantized version of the multimodal model based on Qwen2-VL-7B-Instruct, supporting image-text-to-text tasks with various quantization levels.

EuroLLM-9B-Instruct is a multilingual instruction-following large language model supporting 40+ languages, with special optimization for European language processing capabilities.

This is a quantized version of WizardLM-2-7B using llama.cpp, based on orthogonalized bfloat16 safetensor weight processing, supporting multi-turn dialogues.

GGUF quantized version of DeepSeek-V2-Lite-Chat, supporting multiple quantization types, suitable for local deployment and inference.

Mixtral is a multilingual text generation model based on a Mixture of Experts (MoE) architecture, supporting English, French, Italian, German, and Spanish.